In the vast landscape of mathematical physics and data science, the AᵀCA matrix stands as a universal bridge. Whether you are calculating the displacement of a skyscraper under wind load (Stiffness) or finding the best fit for noisy statistical data (Least Squares), the structure remains the same. When the "perfect" inverse of A fails to exist because the system is singular or overdetermined, the Pseudoinverse A⁺ emerges as our guide back to equilibrium.
1. The Geometry of the Pseudoinverse
The pseudoinverse $A^+$ is an $n$ by $m$ matrix that acts as a perfect inverse where possible. It connects the Four Fundamental Subspaces by ensuring that vectors $u_1, \dots, u_r$ in the column space of $A$ map directly back to $v_1, \dots, v_r$ in the row space.
Mapping Rules
- For $i \leq r$: $A^+ u_i = \frac{1}{\sigma_i} v_i$ (Inverse of the singular value scaling)
- For $i > r$: $A^+ u_i = 0$ (The left nullspace is annihilated)
2. The AᵀCA Construction
Physical systems reach equilibrium through a three-step cycle:
- Kinematics ($Ax=e$): External displacements $x$ create internal strain $e$.
- Constitutive Law ($y=Ce$): Material properties (like Hooke's Law) convert strain to internal stress $y$.
- Equilibrium ($A^Ty=f$): Internal stresses balance external forces $f$.
Combining these yields the master equation: $A^TCAx=f$. If $A^TA$ is invertible, we recover the standard weighted least squares solution.
3. Projections and Identities
Unlike a standard inverse, $AA^+$ and $A^+A$ do not necessarily yield the full Identity matrix. Instead, they act as Projection Matrices:
- $AA^+$ is the projection matrix onto the column space of $A$.
- $A^+ A$ is the projection matrix onto the row space of $A$.
🎯 The SVD Definition
The formal mathematical definition utilizes the Singular Value Decomposition:
$A^+ = V \Sigma^+ U^T = \begin{bmatrix} v_1 \cdots v_r \cdots v_n \end{bmatrix} \begin{bmatrix} \sigma_1^{-1} & & \\ & \ddots & \\ & & \sigma_r^{-1} \\ & & & 0 \end{bmatrix} \begin{bmatrix} u_1 \cdots u_r \cdots u_m \end{bmatrix}^T$
Worked Example: Finding A⁺ for a Rank-1 Matrix
Problem
Consider $A = \begin{bmatrix} 2 & 2 \\ 1 & 1 \end{bmatrix}$. Find $A^+$.
Analysis
The rank $r=1$. The row space is spanned by $v_1 = \frac{1}{\sqrt{2}}(1, 1)$. The column space is spanned by $u_1 = \frac{1}{\sqrt{5}}(2, 1)$. The singular value $\sigma_1 = \sqrt{2^2+2^2+1^2+1^2} = \sqrt{10}$.
Calculation
$A^+ = v_1 \sigma_1^{-1} u_1^T = \frac{1}{\sqrt{2}}\begin{bmatrix} 1 \\ 1 \end{bmatrix} \frac{1}{\sqrt{10}} \frac{1}{\sqrt{5}}\begin{bmatrix} 2 & 1 \end{bmatrix} = \frac{1}{10} \begin{bmatrix} 2 & 1 \\ 2 & 1 \end{bmatrix}$.